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Abstract 

This  project  focused  on  developing  algorithms  for  multi- vehicle,  cooperative 
surveillance  and  reconnaissance  that  operate  in  a  modern  net-centric  environ¬ 
ment*  These  algorithms  take  into  account  the  packet- based  nature  of  modern 
networks  by  coding  information  in  a  manner  that  makes  the  performance  of 
the  system  robust  to  packet  loss,  variable  latency,  and  repeated  transmissions* 
Results  include  analysis  and  design  of  estimation  and  control  algorithms  in 
the  presence  of  packet  loss  and  across  multi-hop  data  networks,  distributed 
estimation  and  sensor  fusion  algorithms  for  networked  environments,  develop¬ 
ment  of  sensor  selection  and  coverage  techniques  for  spatio-temporal  planning, 
and  analysis  of  robustness  to  process  uncertainty  and  computational  node  fail¬ 
ure.  Applications  include  cooperative  estimation,  formation  management,  and 
semi-autonomous  ISR  * 


1  Introduction 

Modern  control  theory  is  largely  based  on  the  abstraction  that  information  (“signals”) 
are  transmitted  along  perfect  communication  channels  and  that  computation  is  either 
instantaneous  (continuous  time)  or  periodic  (discrete  time)*  Modern  communication 
theory  studies  imperfect  channels  but  without  the  strict  limits  on  delays  and  latency 
crucial  in  control*  These  abstractions  have  served  their  fields  well  for  50  years  and 
have  led  to  many  success  stories  in  a  wide  variety  of  applications.  Future  applications 
of  control  in  network-centric  systems  will  be  much  more  information-rich  than  those 
of  the  past  and  will  involve  networked  communications,  distributed  computing,  and 
higher  levels  of  logic  and  decision-making.  New  theory,  algorithms,  and  demonstra¬ 
tions  must  be  developed  in  which  the  basic  input /out put  signals  are  data  packets 
that  may  arrive  at  variable  times,  not  necessarily  in  order,  and  sometimes  not  at 
all.  Networks  between  sensors,  actuation,  and  computation  must  be  taken  into  ac¬ 
count,  and  algorithms  must  address  the  tradeoff  between  accuracy  and  computation 
time*  Progress  will  require  significantly  more  interaction  between  information  theory, 
computer  science,  and  control  than  ever  before. 
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Figure  1:  Cont  rol  architecture  for  a  networked  control  system. 

An  emerging  architecture  for  networked  control  systems  is  shown  in  Figure  1. 
This  architecture  separates  the  traditional  elements  of  sensing,  estimation,  control, 
and  actuation  for  a  given  system  across  a  network  and  also  allows  sharing  of  infor¬ 
mation  between  systems.  Careful  decisions  need  to  be  made  on  how  the  individual 
components  in  this  architecture  are  implemented  and  how  the  communications  across 
the  networked  elements  is  managed.  This  architecture  can  be  used  to  model  either 
a  single  system  (using  either  half  of  the  diagram)  or  multiple  systems  that  interact 
through  the  network. 

2  Accomplishments 

Over  the  three  years  of  this  project,  we  developed  results  in  estimation  and  control 
across  communication  channels  with  limited  bandwidth,  using  simple  models  for  the 
effects  of  packet,  loss. 

2.1  Dynamic  Sensor  Coverage  and  Sensor  Scheduling 

We  have  considered  the  problem  of  active  sensing  using  mobile  sensor  nodes  that  are 
jointly  estimating  the  state  of  a  dynamic  target  as  a  sensor  network  [1],  We  propose 
a  gradient  search-based  decentralized  algorithm  that  demonstrates  the  benefits  of 
distributed  sensing.  We  then  examine  the  task  of  tracking  multiple  targets.  We  use  a 
greedy  algorithm  for  associating  sensors  with  targets  along  with  the  gradient  search 
strategy  for  sensor  vehicle  motion  planning.  Simulation  results  show  that  these  simple 
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decentralized  strategies  perform  quite  well  and  the  sensor  nodes  exhibit  interesting 
cooperative  behavior. 

We  have  also  considered  a  situation  in  which  many  sensors  co-operate  to  estimate 
a  process  [4].  Only  one  sensor  can  take  a  measurement  at  any  time  step.  We  wish 
to  come  up  with  optimal  sensor  scheduling  algorithms.  The  problem  is  motivated 
by  the  use  of  sonar  range-finders  used  by  the  vehicles  on  the  Caltech  Multi- Vehicle 
Wireless  Testbed.  This  problem  involves  searching  a  tree  in  general  and  we  analyze 
two  strategies  for  pruning  the  tree  to  keep  the  computation  limited.  The  first  is  a 
sliding  window  strategy  motivated  by  the  Viterbi  algorithm,  and  the  second  one  uses 
thresholding.  We  also  consider  a  technique  that  employs  choosing  the  sensors  ran¬ 
domly  from  a  probability  distribution  which  can  then  be  optimized.  The  performance 
of  the  algorithms  has  been  illustrated  with  the  help  of  numerical  examples. 

For  the  problem  of  cooperative  surveillance  and  reconnaissance,  the  results  above 
most  be  extended  to  include  decision  on  what  information  is  sent  across  the  network 
(sensor  scheduling)  and  where  the  sensor  are  places  as  a  function  of  time  (sensor 
coverage). 

In  joint  work  with  David  Jeffcoat  at  AFRL/MN  [25],  we  developed  a  theoretical 
framework  for  the  dynamic  sensor  coverage  problem  for  a  simple  case  with  multiple 
discrete  time  linear  dynamical  systems  located  in  different  spatial  locations.  The 
objective  is  to  keep  an  appreciable  estimate  of  the  states  of  the  systems  at  all  times 
by  deploying  a  few  mobile  sensors.  The  sensors  are  assumed  to  have  a  limited  range 
and  they  implement  a  Kalman  filter  to  estimate  the  states  of  ail  the  systems.  The 
motion  of  the  sensor  is  modeled  as  a  discrete  time  discrete  state  Markov  chain.  Based 
on  some  recent  results  on  the  Kalman  filtering  problem  with  intermittent  observations 
by  Sinopoli  et.  ah.  we  derive  conditions  under  which  a  single  sensor  fails  to  solve  the 
coverage  problem.  We  also  give  conditions  under  which  we  can  guarantee  that  a  single 
sensor  is  enough  to  solve  the  dynamic  coverage  problem. 

A  slightly  more  general  set  of  results  was  developed  in  [5],  in  which  we  consider 
the  problem  of  a  set.  of  sensors  is  jointly  trying  to  estimate  a  process.  One  sensor  takes 
a  measurement  at  every  time  step  and  the  measurements  are  then  exchanged  among 
all  the  sensors.  We  .seek  to  find  the  sensor  schedule  that  results  in  the  minimum 
error  covariance  and  describe  a  stochastic  sensor  selection  strategy  that  is  easy  to 
implement  and  is  computationally  tractable.  In  the  sensor  selection  problem,  there  are 
multiple  sensors  that  cannot  operate  simultaneously  (eg,  sonars  in  the  same  frequency 
band).  Thus  measurements  need  to  be  scheduled.  In  the  sensor  coverage  problem, 
a  geographical  area  needs  to  be  covered  by  mobile  sensors  each  with  limited  range. 
Thus  from  every  position,  the  sensors  obtain  a  different  viewpoint  of  the  area  and  the 
sensors  need  to  optimize  their  positions. 


2.2  Estimation  and  Control  with  Information  Loss 

Through  a  sequence  of  results,  we  have  explored  the  performance  of  estimation  algo¬ 
rithms  in  the  presence  of  networked  channels  in  which  information  can  be  lost.  This 
work  builds  on  the  recent  results  of  Sinopoli  et  al.  in  which  they  develop  bounds  on 
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the  performance  of  a  Kalman  filter  in  the  presence  of  packet  loss. 

In  [18],  we  consider  a  discrete  time  state  estimation  problem  over  a  packet* based 
network.  In  each  discrete  time  step,  the  measurement  is  sent  to  a  Kalman  filter 
with  some  probability  that  it  is  received  or  dropped.  The  previous  work  of  Sinopoii 
et  al.  on  Kalman  filtering  with  intermittent  observation  losses  shows  that  there 
exists  a  certain  threshold  of  the  packet  dropping  rate  below  which  the  estimator  is 
stable  in  the  expected  sense.  In  their  analysis,  they  assume  that  packets  are  dropped 
independently  between  all  time  steps.  We  have  developed  a  different  point  of  view 
that  extends  this  analysis  in  two  ways.  First,  we  do  not  required  that,  the  packets 
are  dropped  independently  but  just  that  the  information  gain  wg — defined  to  be  the 
limit  of  the  ratio  of  the  number  of  received  packets  n  during  N  time  steps  as  N  goes 
to  infinity — exists.  Second,  we  show  that  for  any  given  ng,  as  long  as  tt9  >  0.  the 
estimator  is  stable  almost  surely,  i.e.  for  any  given  r,  >  0  the  error  covariance  matrix 
Pit  is  bounded  by  a  finite  matrix  A/,  with  probability  1  —  t.  Given  an  error  tolerance 
M,  7Tg  can  in  turn  be  found.  We  also  give  explicit  formula  for  the  relationship  between 
M  and  c. 

Another  approach  that  we  have  explored  is  the  use  of  multi-description  source 
coding  [12,  13,  11].  In  this  work,  we  split  the  information  that  we  wish  to  send  across 
multiple  packets,  but  encode  the  information  so  that  if  any  collection  of  packets  are 
lost,  the  resulting  information  has  a  known  level  of  distortion.  We  have  considered 
two  cases:  when  the  packet  loss  over  network  links  occurs  in  an  i.i.d.  fashion  or  in  a 
bursty  fashion.  Compared  with  the  traditional  single  description  source  coding,  multi- 
description  (MD)  coding  scheme  can  greatly  improve  the  performance  of  Kalman 
filtering  over  a  large  set  of  packet  loss  scenarios  in  both  cases. 

We  have  also  considered  the  problem  of  (optimal)  control  in  the  presence  of  packet 
loss  [10,  7j.  We  first  prove  a  separation  principle  that  allows  us  to  solve  this  problem 
using  a  standard  LQR  state- feedback  design,  along  with  an  optimal  algorithm  for 
propagating  and  using  the  information  across  the  unreliable  link.  Then  we  present 
one  such  optimal  algorithm,  which  consists  of  a  Kalman  filter  at  the  sensor  side  of  the 
link,  and  a  switched  linear  filter  at  the  controller  side.  Our  design  does  not  assume 
any  statistical  model  of  the  packet  drop  events,  and  is  thus  optimal  for  any  arbitrary 
packet  drop  pattern.  Further,  the  solution  is  appealing  from  a  practical  point  of  view 
because  it  can  be  implemented  as  a  small  modification  of  an  existing  LQG  control 
design. 

We  have  extended  this  work  in  three  ways  to  consider  more  realistic  networks, 
where  information  may  route  through  multiple  nodes  before  being  delivered  to  its 
destination. 

In  the  first  work,  we  consider  the  use  of  multi-hop  protocols  for  improving  the 
convergence  rates  of  consensus  algorithms  [14],  We  propose  multi- hop  relay  protocols 
based  on  the  current  “nearest  neighbor  rules”  consensus  protocols.  By  employing 
multiple-hop  paths  in  the  network,  more  information  is  passed  around  and  each  agent 
enlarges  its  "available”  neighborhood.  We  demonstrate  that  these  relay  protocols 
can  increase  the  algebraic  connectivity  without  physically  adding  or  changing  any 
communication  links.  Moreover,  time  delay  sensitivity  of  relay  protocols  are  discussed 
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in  detail.  We  point  uut  that  a  trade  off  exists  between  convergence  performance  and 
time  delay  robustness.  Simulation  results  are  also  provided  to  verify  the  efficiency  of 
relay  protocols. 

We  have  also  considered  the  problem  of  data  transmissions  over  networks,  where 
each  node  in  the  network  is  allowed  to  perform  some  amount  of  computation  [6], 
We  consider  the  problem  of  determining  the  optimal  processing  at  each  node  in  the 
network  and  provide  a  strategy  that  yields  the  optimal  performance  at  the  cost  of 
constant  memory  and  processing  at  each  node.  We  also  provide  conditions  on  the 
network  for  the  estimate  error  covariance  to  be  stable  under  this  algorithm.  This 
approach  is  applicable  for  networks  of  sensors  that  are  performing  spatio-temporal 
tasks  such  as  cooperative  situational  awareness. 

Finally,  we  have  also  explored  the  design  of  control  strategies  over  lossy  net¬ 
works  [3,  9].  A  network  is  assumed  to  exist  between  the  sensor  and  the  controller  and 
between  the  latter  and  the  actuator.  Packets  are  dropped  according  to  a  Bernoulli 
independent  process,  with  7  and  //  being  the  probabilities  of  losing  an  observation 
packet  and  a  control  packet  respectively,  at  time  any  instant  t,,  A  receding  horizon 
control  scheme  is  proposed  for  the  Linear  Quadratic  Control  (LQG)  problem.  At  each 
instant  N  future  control  inputs  are  sent  in  addition  to  the  current  one.  Under  this 
scheme  the  separation  of  estimation  and  control  is  shown  and  stability  conditions, 
dependent  on  loss  probabilities,  are  provided,  Simulations  show  how  the  overall  per¬ 
formance,  in  terms  of  lower  cost,  increases  with  the  length  of  the  horizon. 

2.3  Distributed  Sensor  Fusion  Using  Dynamic  Consensus 

A  complementary  way  to  explore  distributed  sensor  fusion  is  to  make  use  of  pre¬ 
vious  results  developed  under  AFOSR  MURI  funding  on  consensus  algorithms.  A 
consensus  algorithm  seeks  to  gel  agreement  between  a  set  of  distributed  agents  on  a 
common  quantity.  In  [23]  we  examined  several  dynamical  aspects  of  average  consen¬ 
sus  in  mobile  networks.  The  results  allow  consensus  on  general  time-varying  signals, 
and  allow  tracking  analysis  using  standard  frequency-domain  techniques.  Further, 
the  frequency-domain  analysis  naturally  inspires  a  robust  small-gain  version  of  the 
algorithm,  which  tolerates  arbitrary  non-unifonn  time  delays.  Finally,  we  show  how 
to  exploit  a  dynamical  conservation  property  in  order  t.o  ensure  consensus  tracking 
despite  splitting  and  merging  of  the  underlying  mobile  network. 

This  work  can  be  extended  to  to  obtain  least-squares  fused  estimates  based  on 
spatially  distributed  measurements  [20].  This  mechanism  is  very  robust  to  changes  in 
the  underlying  network  topology  and  performance,  making  it  an  interesting  candidate 
for  sensor  fusion  on  autonomous  mobile  networks.  Examples  have  been  explored  to 
demonstrate  the  the  dependence  of  the  performance  on  the  structure  of  the  under¬ 
lying  network.  A  more  systematic  analysis  of  the  performance  as  various  network 
quantities  such  as  connection  density,  topology,  and  bandwidth  are  varied  lias  also 
been  carried  out  [22],  Our  main  contribution  is  a  frequency-domain  characterization 
of  the  distributed  estimator’s  steady-state  performance;  this  is  quantified  in  terms  of 
a  special  matrix  associated  with  the  connection  topology  called  the  graph  Laplacian, 


and  also  the  rate  of  message  exchange  between  immediate  neighbors  in  the  commu¬ 
nication  network. 

These  results  have  led  to  a  more  general  formulation  of  the  problem  of  distributed 
computing  [19].  The  main  theoretical  contribution  of  this  work  is  a  geometric  formal¬ 
ism  in  which  to  cast  distributed  systems.  This  has  numerous  advantages  and  naturally 
parametrizes  a  wide  class  of  distributed  interaction  mechanisms  in  a  uniform  way.  We 
make  use  of  this  framework  to  present  a  model  for  distributed  optimization,  and  we 
introduce  the  distributed  gradient  as  a  general  design  tool  for  synthesizing  dynamics 
for  distributed  systems.  The  distributed  optimization  model  is  a  useful  abstraction 
in  its  own  right  and  motivates  a  definition  for  a  distributed  extremum.  As  one  might 
expect,  the  distributed  gradient  is  zero  at  a  distributed  extremum,  and  the  dynamics 
of  a  distributed  gradient  How  must  converge  to  a  distributed  extremum.  This  forms 
the  basis  for  a  wide  variety  of  designs,  and  we  are  in  fact  able  to  recover  a  widely 
studied  distributed  consensus  algorithm  as  a  special  case. 

2.4  Robustness  to  Process  Uncertainty  and  Node  Failure 

In  most  of  the  work  on  sensor  coverage  and  distributed  sensor  fusion,  it,  is  assumed 
that  the  participating  vehicles  stay  within  communications  range  of  each  other  and 
maintain  overall  connectivity  of  the  network.  We  have  analyzed  the  feasibility  aspects 
of  motion  planning  for  groups  of  agents  connected  by  a  range-constrained  wireless  net¬ 
work  [21].  Specifically,  we  address  the  difficulties  encountered  when  trajectories  are 
required  to  preserve  the  connectedness  of  the  network.  The  analysis  utilizes  a  quan¬ 
tity  called  the  connectivity  robustness  of  the  network,  which  can  be  calculated  in  a 
distributed  fashion,  and  thus  is  applicable  to  distributed  motion  pitinning  problems 
arising  in  control  of  vehicle  networks.  Further,  these  results  show  that  network  con¬ 
straints  posed  as  connectivity  robustness  constraints  have  minimal  impact  on  reacha¬ 
bility,  provided  that  an  appropriate  topology  control  algorithm  is  implemented.  This 
contrasts  with  more  naive  approaches  to  connectivity  maintenance,  which  can  signif¬ 
icantly  reduce  the  reachable  set. 

In  [17,  16],  we  consider  a  robust  network  control  problem.  We  consider  linear 
unstable  and  uncertain  discrete  time  plants  with  a  network  between  the  sensor  and 
controller  and  the  controller  and  plant.  We  investigate  two  defining  characteristics 
of  network  controlled  systems  and  the  impact  of  uncertainty  on  the  process  dynam¬ 
ics  (modeled  as  parametric  uncertainty).  We  compute  the  minimum  data  rate  and 
minimum  packet  arrival  rate  to  ensure  stability  of  the  dosed  loop  system. 

Another  notion  of  robustness  is  robustness  to  failure  of  individual  computational 
nodes  [8],  This  type  of  uncertainly  has  been  considered  in  the  area  of  distributed 
computation,  but  has  not  been  considered  in  many  of  the  networked  estimation  and 
control  architectures  that  have  been  proposed.  For  a  distributed  algorithm  to  be 
practical,  one  should  be  able  to  guarantee  that  the  task  is  still  satisfactorily  executed 
even  when  agents  fail  to  communicate  with  others  or  to  perform  their  designated 
actions  correctly.  We  present  a  concept  of  robustness  which  is  well-suited  for  general 
distributed  algorithms  for  teams  of  dynamic  agents.  Our  definition  extends  a  similar 
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Figure  2:  Networked  Control  Systems  Architecture. 

notion  introduced  in  the  distributed  computation  literature  for  consensus  problems. 
We  illustrate  the  definition  by  considering  a  variety  of  algorithms  and  identify  possible 
ways  to  make  an  algorithm  robust. 

2,5  Networked  Control  Systems  Architecture 

In  addition  to  the  specific1  research  accomplishments  listed  above,  work  on  this  grant 
has  led  to  the  development  of  a  networked  control  systems  architecture,  illustrated 
in  Figure  2,  that  is  serving  as  the  basis  for  multiple  ongoing  projects  at  Caltech. 
Building  on  the  open  source  Spread  group  communications  protocol,  we  have  devel¬ 
oped  a  modular  software  architecture  that  provides  inter-computer  communications 
between  sets  of  linked  processes.  This  approach  allows  the  use  of  significant  amounts 
of  distributed  computing  for  sensor  processing  and  optimization-based  planning,  as 
well  as  providing  a  very  flexible  backbone  for  building  autonomous  systems  and  fault 
tolerant  computing  systems. 

This  architecture  has  been  interested  in  and  tested  as  part  of  Caltech  s  participa¬ 
tion  in  the  2005  DARPA  Grand  Challenge  [2]. 


3  Transitions 

The  overfill  network  architecture  developed  under  this  grant  was  used  as  part  of 
Caltech’s  entry  in  the  2005  DARPA  Grand  Challenge  and  will  be  used  in  the  2007 
DARPA  Urban  Challenge.  Although  no  direct  effort  of  this  grant  was  applied  to  the 
2005  grand  challenge  (due  to  DARPA  funding  restrictions),  the  overall  framework 
described  above  was  used  as  the  basis  for  our  networked  control  systems  architecture 
and  has  motivated  new  research  directions  of  high  relevance  to  autonomous  vehicles 
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for  Air  Force  missions.  Contact  info:  Richard  Murray  (team  leader),  Caltech,  murray® 
cds.caltech. edu. 
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